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Program Description^ 


PLATO® Achieve Now is a software-based curriculum for the 
elementary and middle school grades. Instructional content is 
delivered via the PlayStation Portable (PSP®) system, allow- 
ing students to access learning materials in various settings. 
Software-based assessments are used to customize individual 


instruction, allowing students to learn at their own pace with 
content appropriate for their skill level. PLATO® Achieve Now is 
aligned with the National Council of Teachers of Mathematics 
guidelines. 


Research^ 


One study of PLATO® Achieve Now meets What Works Clear- 


Based on this one study, the WWC considers the extent 



inghouse (WWC) evidence standards and no studies meet WWC of evidence for PLATO® Achieve Now to be small for math 
evidence standards with reservations. The one study Included achievement. 

1,037 sixth-grade students in 13 schools In three districts across 
the United States.® 



Eff6CtiV6neSS PLATO® Achieve Now was found to have no discernible effects on math achievement. 



Math Achievement 
Rating of effectiveness No discernible effects 
Improvement index'^ Average: -1 percentile point 

1. The descriptive information for this program was obtained from publiciy avaiiabie sources: the program’s website {http://www.piato.com/Products/ 
PLATO-Achieve-Now-on-PSP/PLATO-Achieve-Now-on-PSP-Mathematics.aspx, downioaded June 2009) and Campuzano et. ai. (2009). The WWC 
requests developers to review the program description sections for accuracy from their perspective. Further verification of the accuracy of the descrip- 
tive information for this program is beyond the scope of this review. 

2. The studies in this report were reviewed using WWC Evidence Standards, Version 1.0 (see the WWC Standards). 

3. The evidence presented in this report is based on avaiiabie research. Findings and conciusions may change as new research becomes avaiiabie. 

4. These numbers show the average and range of student-ievei improvement indices for ali findings across the study. 
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The studies Campuzano, Dynarski, Agodini, and Rail (2009) and 
Dynarski et. al. (2007), cited in the references section below, 
were prepared, in whole or in part, by staff of Mathematica 
Policy Research (MPR). Because the deputy principal investiga- 
tor for the WWC review of Middle School Math is also a MPR 



staff member, the study was rated by staff members from other 
organizations, who also prepared the intervention report. The 
report was then reviewed by the principal investigator, a WWC 
Quality Assurance reviewer, and an external peer reviewer. 



Developer and contact 

PLATO® Achieve Now was developed and is distributed by 
PLATO Learning, Inc. Address: 10801 Nesbitt Ave. South Bloom- 
ington, MN 55437. Email: marketing@plato.com. Web: http:// 
www.plato.com. Telephone: (800) 44-PLATO (800-447-5286). 

Scope of use 

No scope of use statistics are available. 

Teaching 

The instructional content of PLATO® Achieve Now is delivered 
via the PlayStation Portable (PSP®) system. The program is 
designed to deliver instructional content to students in various 
contexts, including during the school day, in after-school pro- 
grams, and at home. 

PLATO® Achieve Now focuses on pre-algebraic concepts 
and includes content pertaining to rational numbers in related 
organizational patterns, proportion and percent, integers, prob- 
ability, statistics, problem solving, geometry, measurement, and 



the foundational concepts of algebra I. PLATO® Achieve Now is 
designed for the elementary and middle school grades. 

Students use the product for independent practice and rein- 
forcement of math skills. The curriculum utilizes an individualized, 
competency-based mastery model to guide student progression 
through the materials. The instructional aspects of the program 
include interactive games, as well as school and home learning 
activities. Software-based assessments are used to customize 
instruction to individual students. Based on the assessments, stu- 
dents work at their own pace on activities identified by the teacher, 
with content appropriate for their skill level. Recommended usage is 
30 minutes per day, four days a week, for at least 10 weeks. 

Teachers receive training through web-based meetings and 
online self-tutorials. Ongoing support during the school year is 
also provided. PLATO® Achieve Now is aligned with the National 
Council of Teachers of Mathematics guidelines. 

Cost 

Contact PLATO Learning, Inc. for pricing information. 



Thirteen studies reviewed by the WWC investigated the effects 
of PLATO® Achieve Now. One study (Campuzano, Dynarski, 
Agodini, & Rail, 2009) is a randomized controlled trial that meets 
WWC evidence standards. The remaining 12 studies do not meet 
either WWC evidence standards or eligibility screens. 

Meets evidence standards 

Campuzano, Dynarski, Agodini, and Rail (2009) investigated 
the effect of PLATO® Achieve Now on math achievement in 13 
schools in three districts that randomly assigned teachers and 



their classes to the PLATO® Achieve Now intervention group 
(regular classroom instruction supplemented with the PLATO® 
Achieve Now intervention) or to the control condition during 
the 2004-2005 or 2005-2006 school year. The study included 
1,037 sixth-grade regular education students (547 PLATO® 
Achieve Now and 490 control) taught by 39 teachers (21 PLATO® 
Achieve Now and 18 control). Of these 39 teachers, 19 were in 
the study for 2004-2005 only, 18 teachers were in the study for 
2004-2005 and 2005-2006, and 2 teachers were in the study 
for 2005-2006 only. Student test scores in classrooms with 
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Research (continued) 



Effectiveness 



The WWC found PLATfP 
Achieve Now to have no 
discernible effects for math 
achievement 



WWC Intervention Report PLATO' 



teachers supplementing instruction with PLATO® Achieve Now 
were compared with student test scores in ciassrooms with 
teachers in the controi condition, who taught math as they would 
normally, which could include technology products already avail- 
able to those teachers. 

Extent of evidence 

The WWC categorizes the extent of evidence in each domain 
as small or medium to large (see the WWC Procedures and 



Standards Handbook, Appendix G). The extent of evidence takes 
into account the number of studies and the total sample size 
across the studies that meet WWC evidence standards with or 
without reservations.® 

The WWC considers the extent of evidence for PLATO® 
Achieve Now to be small for math achievement. 



Findings 

The WWC review of interventions for Middle School Math 
addresses student outcomes in the math achievement domain. 
The findings below present the authors’ estimates and WWC- 
calculated estimates of the size and the statistical significance of 
the effects of PLATO® Achieve Now on students.® 

Math Achievement 

Campuzano, Dynarski, Agodini, and Rail (2009) reported nega- 
tive but not statistically significant effects of PLATO® Achieve 
Now on math achievement based on the Stanford Achievement 
Test-Tenth Edition (SAT-10), the Iowa Test of Basic Skills (ITBS), 
and the New Mexico Standards Based Assessment (NMSBA).^ 
The authors reported all effects in normal curve equivalent (NCE) 
units. The WWC confirmed that these negative effects were 
neither statistically significant nor large enough to be considered 

Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and an 



substantively important by WWC criteria (an effect size of at 
least 0.25). 

In sum, in the math achievement domain, the WWC reviewed 
findings from one study which showed indeterminate effects. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings, the size of 
the difference between participants in the intervention and the 
comparison conditions, and the consistency in findings across 
studies (see the WWC Procedures and Standards Handbook, 
Appendix E). 



average improvement index across studies (see WWC Proce- 
dures and Standards Handbook, Appendix F). The improvement 
index represents the difference between the percentile rank 
of the average student in the intervention condition and the 



5. The extent of evidence categorization was developed to tell readers how much evidence was used to determine the Intervention rating, focusing on 
the number and size of studies. Additional factors associated with a related concept— external validity, such as the students’ demographics and the 
types of settings in which studies took place— are not taken into account for the categorization. Information about how the extent of evidence rating 
was determined for PLATO® Achieve Now is in Appendix A5. 

6. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within 
classrooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch. For the formulas the WWC used to 
calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Stan- 
dards Handbook, Appendix D for multiple comparisons. In the case of Campuzano, Dynarski, Agodini, and Rail (2009), no corrections for clustering 
or multiple comparisons were needed. 

7. All three exams were not used in each district; rather, districts used at least one of the three exams as the pretest and/or posttest measure. 
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The WWC found PLAT(P 
Achieve Now to have no 
discernible effects for math 
achievement (continued) 



References 



percentile rank of the average student in the comparison condi- 
tion. Uniike the rating of effectiveness, the improvement index is 
entireiy based on the size of the effect, regardless of the statisti- 
cal significance of the effect, the study design, or the analysis. 
The improvement index can take on values between -50 and 
+50, with positive numbers denoting favorable results for the 
intervention group. 

The average improvement index for math achievement is -1 
percentile point based on the sole finding from one study. 



Summary 

The WWC reviewed 13 studies on PLATO® Achieve Now. One of 
these studies meets WWC evidence standards; the remaining 12 
studies do not meet either WWC evidence standards or eligibility 
screens. Based on the one study, the WWC found no discern- 
ible effects for the math achievement domain. The conclusions 
presented in this report may change as new research emerges. 



Meets WWC evidence standards 

Campuzano, L., Dynarski, M., Agodini, R., & Rail, K. (2009). 
Effectiveness of reading and mathematics software 
products: Findings from two student cohorts (NCEE 
2009-4041). Washington, DC: National Center for Education 
Evaluation and Regional Assistance, Institute of Education 
Sciences, U.S. Department of Education. 

Additional source: 

Dynarski, M., Agodini, R., Heaviside, S., Novak, T., Carey, N., 
Campuzano, L., Means, B., Murphy, R., Penuel, W., Javitz, 
H., Emery, D., & Sussex, W. (2007). Effectiveness of reading 
and mathematics software products: Findings from the 
first student cohort. Washington, DC: U.S. Department of 
Education, Institute of Education Sciences. 

Studies that fall outside the Middle School Math review 
protocol or do not meet WWC evidence standards 

PLATO Learning, (n.d.). Evaiuation study: Camden City Schooi 
District. Retrieved April 23, 2008, from http://www.plato. 
com/media/Evaluation%20Studies/C/Camden%20City%20 
School%20District.pdf. The study is ineligible for review 
because it does not use a comparison group. 

PLATO Learning, (n.d.). Evaiuation study: Carver Middie Schooi: 
Chicago Pubiic Schooi District. Retrieved April 23, 2008, 
from http://www.plato.eom/media/Evaluation%20Studies/C/ 
Carver%20Middle%20School.pdf. The study is ineligible for 
review because it does not use a comparison group. 



PLATO Learning, (n.d.). Evaiuation study: De Anza Junior High 
Schooi. Retrieved April 23, 2008, from http://www.plato.com/ 
media/Evaluation%20Studies/D/De%20Anza%20Junior%20 
High%20School.pdf. The study is ineligible for review because 
it does not use a comparison group. 

PLATO Learning, (n.d.). Evaiuation study: Eiectronic ciassroom 
of tomorrow. Retrieved April 23, 2008, from http://www. 
plato.eom/media/Evaluation%20Studies/E/Electronic%20 
Classroom%20of%20Tomorrow.pdf. The study is ineligible for 
review because it does not use a comparison group. 

PLATO Learning, (n.d.). Evaiuation study: Frederick Dougiass 
Academy. Retrieved April 23, 2008, from http://www. 
plato.eom/media/Evaluation%20Studies/F/Fredrick%20 
Douglass%20Academy.pdf. The study is ineligible for review 
because it does not use a comparison group. 

PLATO Learning, (n.d.). Evaiuation study: Joshua Middie Schooi. 
Retrieved April 23, 2008, from http://www.plato.com/media/ 
Evaluation%20Studies/J/Joshua%20Middle%20School.pdf. 
The study is ineligible for review because it does not use a 
comparison group. 

PLATO Learning, (n.d.). Evaiuation study: Paterson Pubiic 
Schoois. Retrieved April 29, 2008, from http://www.plato. 
com/media/Evaluation%20Studies/P/Paterson%20Public%20 
Schools.pdf. The study is ineligible for review because it does 
not use a comparison group. 

PLATO Learning, (n.d.). Evaiuation study: Rod Paige Middie 
Schooi. Retrieved April 23, 2008, from 
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Rod%20Paige%20Middle%20School.pdf. The study is 
ineligible for review because it does not use a comparison 
group. 

PLATO Learning, (n.d.). Evaluation study: Saluda Middle School. 
Retrieved April 23, 2008, from http://www.plato.com/media/ 
Evaluation%20Studies/S/Saluda%20Middle%20School.pdf. 
The study is ineligible for review because it does not use a 
comparison group. 

PLATO Learning, (n.d.). Evaluation study: Schrum Memorial 
Middle School. Retrieved April 23, 2008, from http://www. 
plato.eom/media/Evaluation%20Studies/S/Schrum%20 



Memorial%20Middle%20School.pdf. The study is ineligible for 
review because it does not use a comparison group. 

PLATO Learning, (n.d.). Evaluation study: Wilson Junior High 
School. Retrieved April 23, 2008, from http://www.plato. 
com/media/Evaluation%20Studies/W/Wilson%20Junior%20 
High%20School.pdf. The study is ineligible for review because 
it does not use a comparison group. 

Poore, J. H., & Hamblen, J. W. (1983). Improvement of basic 
mathematical skills with PLATO: An experiment. Association 
for Educational Data Systems Journal, 24, 224-259. The study 
is ineligible for review because it does not use a comparison 
group. 
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Appendix 



Appendix A1 Study characteristics: Campuzano, Dynarski, Agodini, & Raii, 2009 (randomized controiied triai) 



Characteristic 


Description 


Study citation 


Campuzano, L., Dynarski, M., Agodini, R., & Rail, K. (2009). Effectiveness of reading and mathematics software products: Findings from two student cohorts (NCEE 
2009-4041). Washington, DC: Nationai Center for Education Evaiuation and Regionai Assistance, institute of Education Sciences, U.S. Department of Education. 

Additional source: 

Dynarski, M., Agodini, R., Eleaviside, S., Novak, T., Carey, N., Campuzano, L., Means, B., Murphy, R., Penuei, W., Javitz, H., Emery, D., & Sussex, W. (2007). 
Effectiveness of reading and mathematics software products: Findings from the first student cohort. Washington, DC: U.S. Department of Education, institute of 
Education Sciences. 


Participants 


The study sampie inciuded 1,037 sixth-grade students (547 PLATCP Achieve Now, 490 controi) taught by 39 teachers (21 PLATCP Achieve Now, 18 controi) in 13 schoois 
across three districts in muitipie states across the country during the 2004-05 and 2005-06 schooi years. Of the study sampie, approximateiy 53% were femaie (52% 
PLATCP Achieve Now anti 55% controi), 74% received free or reduced-price iunch (not reported by intervention status), 42% were Hispanic (not reported by intervention 
status), and 40% were African-American (not reported by intervention status). Approximateiy 80% of the teachers in the study were femaie (81% PLATO^ Achieve Now, 78% 
controi) with an average of 11 years of teaching experience (9 years PLATCP Achieve Now, 13 years controi) and 33% of whom obtained a master’s degree (24% PLATO® 
Achieve Now; 44% controi). 


Setting 


The study took piace in 13 schoois in three districts in muitipie states across the country. 


Intervention 


Students were taught using PLATO® Achieve /Vow during the 2004-05 and/or 2005-06 schooi years. PLATO® Achieve /Vowsuppiemented standard mathematics instruction 
for the treatment group. According to the study authors, PLATO® Achieve /Vow students used the product for independent practice and reinforcement of math skiiis. Students 
worked at their own pace on activities identified by the teacher. According to the authors, the recommended usage is 30 minutes per day, four days per week, for at ieast 10 
weeks. 


Comparison 


Comparison students were taught in traditionai ciasses, with the teachers utiiizing any technoiogy products aiready avaiiabie to them. 


Primary outcomes 
and measurement 


The primary outcome measures in Year 2 of the study were the Stanford Achievement Test-Tenth Edition (SAT-1 0), the iowa Test of Basic Skiiis (iTBS), and the New Mexico 
Standards Based Assessment (NMSBA). Districts in this study used at ieast one of these three exams as the outcome measure to obtain pretest and/or posttest scores. The 
study authors converted the scaie scores from these tests to normai curve equivaient (NCE) scores with a range of 1 to 99 and an average of 50 to standardize the measures 
across tests and cohorts. For a more detaiied description of these outcome measures, see Appendix A2. 


Staff/teacher training 


Vendor training sessions generaiiy took piace in host districts, and sometimes host schoois, during the summer or eariy faii of 2004. The initiai training iasted about 6 hours 
and varied by product from 4 hours to about 8 hours. Vendors deiivered ongoing support in severai modes. Product representatives visited teachers; vendors aiso provided 
support through emaii, teiephone heip desks, and additionai training at schoois. 
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Appendix A2 Outcome measures for the math achievement domain 



Outcome measure 


Description 


Stanford Achievement 
Test-Tenth Edition (SAT-10) 


The SAT-1 0 is a commercially available standardized test that measures number sense and operations, patterns, relationships, algebra, geometry, measurement, data, statis- 
tics, and probability. The study authors converted the scale scores from the SAT-10 math test to normal curve equivalent scores to standardize the measures across tests and 
cohorts (as cited in Campuzano, Dynarski, Agodini, & Rail, 2009). 


Iowa Test of Basic 
Skills (ITBS) 


The ITBS is a group-administered, norm-referenced battery of achievement tests for students in kindergarten through eighth-grade. The tests are ordered by levels ranging 
from 5 to 14, which correspond to the targeted age of the students. The sixth-grade level of the math test measures number properties and operations, algebra, geometry, 
measurement, probability and statistics, and estimation. The study authors converted the scale scores from the ITBS math test to normal curve equivalent scores to standard- 
ize the measures across tests and cohorts (as cited in Campuzano, Dynarski, Agodini, & Rail, 2009). 


New Mexico Standards 
Based Assessment (NMSBA) 


The NMSBA is a criterion-referenced test. The math section measures number and operations, algebra, geometry, measurement, data analysis, and probability. Items were 
aligned to the New Mexico K-12 content standards, benchmarks, and performance standards. The study authors converted the scale scores from the NMSBA math test to 
normal curve equivalent scores to standardize the measures across tests and cohorts (as cited in Campuzano, Dynarski, Agodini, & Rail, 2009). 
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Appendix A3 Summary of study findings inciuded in the rating for the math achievement domain^ 









Authors’ findings from the study 














Mean outcome 
(standard deviation)^ 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
students) 


PLATCF Achieve 

Now Comparison 

groups group’* 


Mean 

difference^ 
(PLATO® 
Achieve Now- 
comparison) 


Statistical 

Effect significance^ 

size® (at a = 0.05) 


Improvement 

index® 



Campuzano, Dynarski, Agodini, & Rail, 2009 (randomized controlled trial)^ 



Math NCE*'’ Grade 6 


39/1,037 


50.09 

(18.44) 


50.67 

(19,38) 


-0,58 


-0,03 


ns 


-1 


Domain average for math achievement** 










-0.03 


na 


-1 



ns = not statistically significant 

na = not applicable 

NCE = normal curve equivalent 



1 . This appendix reports findings considered for the effectiveness rating and the average improvement indices for the math achievement domain. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. The intervention group value is the control group mean plus the program coefficient from the hierarchical linear model (HLM) analysis. 

4. The control group mean is the unadjusted control group mean. 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

6. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting favorable results for the intervention group. 

9. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Campuzano, Dynarski, Agodini, & Rail 
(2009), no corrections for clustering or multiple comparisons were needed. 

10. The study authors converted the scale scores from the SAT-10, ITBS, and NMSBA tests to normal curve equivalent units to standardize measures across tests and cohorts. NCE scores have a 
mean of 50, a standard deviation of 21.06, and a range of 1 to 99. A single overall NCE value is reported by the authors to reflect the average annual treatment effect across all districts. 

11 . This row provides the study average, which in this instance, is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The 
domain improvement index is calculated from the average effect size. 
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Appendix A4 PLATO^ Achieve Now rating for the math achievement domain 

The WWC rates an intervention’s effects in a given outcome domain as positive, potentiaiiy positive, mixed, no discernible effects, potentiaiiy negative, or negativeJ 
For the outcome domain of math achievement, the WWC rated PLATO® Achieve Now as having no discernible effects. 



Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: None of the studies shows a statisticaiiy significant or substantively important effect, either positive or negative. 

Met. The one study did not show a statisticaiiy significant or substantively important effect either positive or negative. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statisticaiiy significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed statisticaiiy significant positive effects. One study met WWC evidence standards for a strong design. 

AND 

• Criterion 2: No studies showing statisticaiiy significant or substantively important negative effects. 

Met. No studies showed statisticaiiy significant or substantively important negative effects. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At ieast one study showing a statisticaiiy significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantiveiy important positive effect. 

AND 

• Criterion 2: No studies showing a statistically significant or substantiveiy important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statisticaiiy significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important negative effect, no studies showed indeterminate effects, and no 
studies showed statisticaiiy significant or substantively important positive effects. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At ieast one study showing a statisticaiiy significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantiveiy important negative effect, but no more such studies than the number showing a statisticaiiy significant or substantiveiy important pos/f/Ve effect. 

Not met. No study showed a statisticaiiy significant or substantively important effect, either positive or negative. 

OR 

1 . For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Flandbook, Appendix E. 



(continued) 
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Appendix A4 PLATO^ Achieve Now rating for the math achievement domain (continued) 



• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No study showed a statistically significant or substantively important effect, either positive or negative. 

Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important negative effect. 

Not met. No studies showed a statistically significant or substantively important negative effect. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important pos/f/Ve effect, or more studies showing statistically significant or substantively 
important negative effects than showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important positive effect. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant or substantively important negative effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important positive effect. 
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Appendix A5 Extent of evidence by domain 



Outcome domain 


Number of studies 


Schools 


Sample size 


Students 


Extent of evidence^ 


Math achievement 


1 


13 




1,037 


Small 



1 . A rating of “medium to iarge” requires at ieast two studies and two schoois across studies in one domain and a totai sampie size across studies of at ieast 350 students or 14 ciassrooms. Other- 
wise, the rating is “smaii.” For more detaiis on the extent of evidence categorization, see the WWC Procedures and Standards Handbook, Appendix G. 
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